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Abstract 

Wc study the existence of dual certificates in convex minimization problems where a matrix 
Xo is to be recovered under semidefinite and linear constraints. Dual certificates exist if and only 
if the problem satisfies strong duality and the dual problem attains its optimum. Because the 
dual problem may fail to attain its optimum, and near-optimal dual variables may necessarily 
diverge, optimization algorithms that rely on convergencee of dual variables lose their theoretical 
foundation. In the case that X is rank 1, we prove that dual certificates are guaranteed to 
exist if the linear measurement matrices can not be recombined to form something positive 
and orthogonal to Xq. If the measurements can be recombined in this way, the problem is 
equivalent to one that is 'completed' with additional linear constraints. That completed problem 
is guaranteed to have a dual certificate at the minimizer. If a set of measurement matrices is 
not complete in this way, there is a convex problem for which a dual certificate does not exist 
at a minimizer. 

1 Introduction 

We consider the problem of showing that Xo is a minimizer to the semidefinite program 

min /(X) subject to X h 0, A(X) = b, (1) 

where X S Kgym i s a symmetric and real- valued matrix, / is convex and continuous everywhere, A is 
linear, and -4(X ) = b € R m . Let (X, Y) = tr(Y*X) be the Hilbert-Schmidt inner product. Matrix 
orthogonality is understood to be with respect to this inner product. The linear measurements 
„4(X) = b can be written as 

A(X)i = (X, Aj) = hi for i = 1, . . . , m, 

for certain symmetric matrices A^. Note that the adjoint of A is given by A*X = 2~Z^=i ^i-^-i- 

A case of particular interest is when Xo = xoXg is rank one. For example, the vector phase 
retrieval problem motivated by X-ray crystallography can be written as a rank one matrix recovery 
problem through a process called lifting [4]. That matrix recovery problem can be solved through 
PhaseLift [7], where /(X) = tr(X) and Aj = zz* for vectors Zj. This problem can also be viewed 
as a feasibility problem, where /(X) = [8, 5j. Another example is the compressive phase retrieval 
problem [TTJ [12], where Xo is also presumed to be sparse. In this case /(X) = ||X||i + c tr(X), 
where the first term is the entry- wise C 1 norm of X. These programs are all semidefinite instances 
of low-rank matrix recovery problems [6j [TO] . 
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1.1 Convex Optimization Background 

A common method of proving that a particular element Xo is a minimizer to matrix recovery 
problems is certification [101 QTJ CO • We call the matrix Y a certificate of optimality if 

(Y, X - X ) < for A(X) = b, X y 0, 
<Y,X-X ) >0for /(X)</(X ). 

That is, a certificate acts as the normal vector to a hyperplane that separates the feasible set from 
the set of strictly smaller objectives. By the separating hyperplane theorem, such a certificate exists 
if and only if Xo is a minimizer. By the mild assumptions on / (which establish it is Lipschitz on 
bounded sets), such a certificate can be taken to be of the form Y £ —df(Ko), where df is the 
subdifferential of /. 

In many cases, the certificate can be represented in terms of the problem's dual variables. As 
standard in dual formulation of a convex program [3], the Lagrangian of (TlJ) is given by 

£(X, A, Q) = /(X) + (A, A(X) - b) + (Q, X). 

where Q € ^ and A £ W 71 . The dual variables (A, Q) are dual-feasible when Q ^ 0. We note 
that the subdifferential of C with respect to X and evaluated at Xo is 8C(X.q, A, Q) = 9/ (Xo) + 
A*X + Q. The Lagrangian dual function is defined by g(\, Q) = infx £(X, A, Q). 
The dual problem for ([IJ) is 

sup g(X,Q). (2) 

Let p* and d* be the optimal values of (UJ and ([2]), respectively. The duality gap is p* — d*, which 
is always nonnegative. Problem (pQ) satisfies strong duality when the duality gap is zero. 
We call (A, Q) a dual certificate at Xo if 

A*\ + Qe-df(X ), (3) 
Q^0,Q±X . (4) 

That is, a dual certificate (A, Q) defines a certificate of optimality, is dual feasible, and satisfies 
the complementary slackness condition. Sometimes, we will refer to Y = A*X + Q as the dual 
certificate. To simplify notation, let the cone of candidate dual certificates be 

S = {A*X + Q | Q r< 0,Q J_ X }. (5) 

That is, S is the direct sum of the range of A* with the negative semidefinite cone orthogonal to 
X . 

Theorem 1. If (A, Q) is a dual certificate at Xo, then Xo is a minimizer to (pQ). 

By elementary arguments from convex optimization, a dual certificate is given by the optimizer of 
the dual problem. 

Theorem 2. Let Xo be a minimizer of ([1]). The following are equivalent: 

(a) (A, Q) is a dual certificate at Xo, 

(b) (A, Q) is dual optimal and strong duality holds. 

For the sake of completeness, we provide a proof of these elementary theorems in Section [2l 
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1.2 Examples without Dual Certificates 

We provide two examples of semidefinite problems for which a dual certificate does not exist at a 
minimizer. To aid the analysis, let ej be the ith standard basis vector, let y <g> q = yq* + qy* be 
the symmetric tensor product, let ||X||f be the Frobenius matrix norm, and note that 

X h and (X, qq*) = for q € R n Xq = =► (X, y ® q) = for any y G R n . (6) 

Example 1. Let Xo = eie*. There is no dual certificate at Xo for the problem 

mm^\\X\\l subject to XhO,(x,r Q ° U = 0,/x Y| |)) = 1- (7) 

Further, this problem satisfies strong duality where the dual optimum is not achieved. 

By ([6]), any feasible X for Example [T] satisfies (X,ei <8> = 0. Hence, the minimizer and only 
feasible point of is X . The subdifferential of /(X) = i||X||| is 9/(Xo) = {Xo}. We note that 
there is no dual certificate because there is no (A, Q) satisfying Q ■< 0, Q _L Xo, and 

1 0\ . /0 0\ . (I l\ _ 

o o =Al o i +Aa i i +Q - 



The dual problem for ([7]) is 



1 

sup -- 



0\ . (I 1\ _ 

o i +A2 1 1 +Q 



x?. 



Taking 



A«-r i+ _i /e ),Q«-(r _; /£ 



we see that 9 (A( £ ),Q( £ )) -> | = p*. Hence, there is no duality gap, and by Theorem [1] the dual 
optimum is not achieved. Further, any sequence of dual- feasible variables approaching the dual 
optimum necessarily diverges. To see this, consider g(X {e) , Q (e) ) -> d* = p*. If both and Q( £ ) 
have a bounded subsequence, then there is a further subsequence that converges. By continuity of 
g for this example, there would be a dual optimal (A, Q), which is a contradiction. 

Note that if ([7]) were supplemented with the constraint (X, ei <8> e2) = 0, there would exist a 
dual certificate at Xo- 

Example 2. Let Xo = e\e*. There is no dual certificate at Xo for the problem 
min /x, ft Jj \ ^6jeci to X ^ 0, /x, (° \ = 0. 

Further, this problem does not satisfy strong duality. 

For any (A, Q) with Q ■< 0, g(X, Q) = — oo. Hence the duality gap for this problem is infinite 
and strong duality does not hold. 
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1.3 Strong Duality and Semidefinite Programs 

The lack of strong duality or dual attainment in semidefinite programs is well known in the literature 
[9]. It can be avoided when certain constraint qualifications hold. For example, Slater's condition 
requires the existence of a strictly feasible point in order to get strong duality [3]. A generalized 
Slater condition [2] requires a point in the relative interior of the cone of feasible points. In the 
present paper, we are interested in rank one solutions, which lie on the boundary of the semidefinite 
cone. Hence, we do not appeal to Slater-like conditions. 

Without a constraint qualification, it is possible to achieve strong duality by regularizing the 
primal problem or considering an alternative to the Lagrangian dual. One method of regularization 
involves rewriting the primal problem in terms of the minimal cone [2j [2]. Alternative to the 
Lagrangian dual is the extended Lagrange-Slater Dual |13} 114] . which can be written down in 
polynomial time. 

1.4 Main Results 

In this paper, we present a constraint qualification for the problem ([1]) to satisfy strong duality 
and dual attainment. This condition is different from Slater-like conditions because it does not 
explicitly consider the set of feasible points. Instead it focuses on whether or not there is any 
combination of measurement matrices that positive and orthogonal to Xq. 

1.4.1 Sufficient condition for dual certificate existence 

The lack of a dual certificate in the examples arrives because there is a linear combination of Aj that 
is positive semidefinite and orthogonal to Xo- If this case is excluded, a dual certificate necessarily 
exists at the rank-one solution Xo- 

Theorem 3. // Xo = xoXq minimizes ([I]) and there does not exist a nonzero A € span{ Aj}^!l 1 
such that A y and A _L Xo, then there exists a dual certificate at Xo- 

For some measurement matrices Aj of practical interest, Theorem[3]is applicable. For example, 
the following corollary implies a dual certificate exists with probability 1 if Xo = xoXq minimizes 
(PQ) with Ai = ZiZ* for i.i.d. Gaussian and m < n. This is the case arising in the compressive 
phase retrieval problem 

Corollary 1. //Xo = xoXq minimizes ([1]) and {AjXo}™ =1 are independent, then there exists a dual 
certificate at Xo- 

1.4.2 Completeness condition implies dual certificate existence 

If there is a positive semidefinite measurement matrix A € span{Aj} that is orthogonal to Xo, then 
([6]) provides additional constraints on X that may or may not be implied by A(X.) = b alone. For 
any eigenvector q corresponding to a positive eigenvalue of A, and for any y, all feasible X satisfy 
(X, y <S> q) = 0. If each of these y ® q already belong to the cone of candidate dual certificates S, 
then we consider the measurements 'complete', and we prove a dual certificate necessarily exists. 

Theorem 4. // Xo = xoXq minimizes ([I]) and S = {.4* A + Q | Q -< 0, Q _L Xo} satisfies the 
completeness condition 

qq* G S and q _L x => y <g> q € S Vy, (8) 
then there exists a dual certificate at Xq . 
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We note that completeness of S does not imply that y ®q £ span {Aj}. That is, (X, y ® q) = 
need not be implied by -A(X) = b. Instead, completeness requires that y<8>q differs from spanjA^} 
by something negative. 

1.4.3 Completion process 

If the problem (pQ) does not satisfy the completeness condition (jSJ), an optimality certificate for fJTf) 
can be expressed as a dual certificate for the problem augmented with linear constraints implied 
by X y 0, A(X.) = b. This completed problem is equivalent to ([T]) and its existence is a corollary 
of Theorem HJ 

Corollary 2. //Xq = xqXq minimizes (fl]), then there exists an equivalent equivalent problem 



such that there exists a dual certificate at Xo- This problem is equivalent to ([I]) in the sense that 
X h and A(X) = b O X y and .A(X) = b. 

The following procedure outlines how to complete the set of measurement matrices {A;} in 
order to obtain an equivalent problem satisfying the completeness condition (jHJ): 

1. Consider all A y 0, A € span{Aj}, (A, X ) = 0. 

2. Write each A = J2k c kQ.kcCk with > 0. 

3. For every j, if e^ (8> q^ ^ span{Aj}, append (X, e 3 - <8> q^} = to X(X) = b. 

4. Repeat until A remains unchanged. 

This process will terminate after finitely many repetitions because dim(span{Aj}) increases 
each time. Because the resulting A will satisfy (jSJ), we apply Theorem H] and have thus proven 
Corollary [2j Unfortunately, the semidefinite feasibility problem implicit to step 1 is of unknown 
computational complexity |13] , Hence, this procedure is of limited computational use. 

1.4.4 Weak necessity of the completeness condition 

If the measurement matrices fail to satisfy the completeness condition (|8|) , a particular problem of 
form (pQ) may or may not have a dual certificate at a minimizer Xo- Nonetheless, there is a problem 
for which a dual certificate does not exist. 

Theorem 5. Fix Xo = xoXq and the matrices {Aj}™ 1 . If S does not satisfy the completeness 
condition (|8|), there exists a convex problem ([T|) such that Xo is a minimizer and a dual certificate 
does not exist at Xo- 

This weak form of necessity of the completeness condition arises in part because of an equivalence 
between completeness and the additivity of subgradients of indicator functions over the constraints, 
as proven in Lemma [TJ 




(9) 
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1.4.5 Implication for numerical optimization 

As illustrated by Example HJ semidefinite problems that fail the completeness condition ([5]) may not 
have dual optimal variables. It can be the case that dual variable diverge whenever g(\( e \ Q^) — > 
d*, even when strong duality holds. Hence, any numerical scheme that explicitly represents (A, Q) 
can not be convergent in these variables. Such dual methods may still provide reasonable solutions 
and estimates of the duality gap. Nonetheless, if they require convergence in dual variables, their 
theoretical foundation becomes unclear. Also, any dual method becomes prone to large relative 
errors when forming quantities like Y = ^4*A + Q. The possible divergence of dual variables can be 
avoided by implementing a completion process like in Section 11.4.3} or by considering an alternative 
dual formulation, such as the extended Lagrange-Slater dual |14J. 



1.5 Organization of this paper 

In Section [1.6l we present the notation used throughout the paper. In Section^ we prove Theorems 
[1] and [2] which are elementary results from convex optimization. In Section [31 we prove Theorems 
[3HS] and Corollary [TJ Corollary [2] was proven in Section 11.4.31 The proofs of Theorems [3] and [J] rely 
on technical lemmas concerning the additivity of subdifferentials of indicator functions, and on the 
closedness of S. These lemmas are proven in Section [H 



1.6 Notation 

Let Kgy^ be the space of symmetric, real-valued matrices of size n. Matrices will be denoted with 
boldface capital letters, and vectors will be denoted with boldface lowercase letters. Let (•, •} be the 
usual inner product for vectors and the Hilbert-Schmidt inner product for matrices. Let x ® y = 
xy* + yx* be the symmetric tensor product. For a subspace V C R n , let V 1 - be the orthogonal 
complement with respect to the ordinary inner product. Let Iyj_ be the matrix corresponding to 
orthogonal projection of vectors onto V^. Let Vy±X. = Iy±~KI v ± be the projection of symmetric 
matrices onto symmetric matrices with row and column spans in V-^. Let V x ± be the special case 
in the instance where V = spanjxo}. In the special case where xo is the coordinate basis element 
ei, V ±~X. is the restriction of X to the lower-right n-lxn-1 block. 
Let the indicator function for the set be 



'n(X) 




if X € SI, 

if x ^ a 



2 Proofs of Background Theorems [T] and [2] 

Theorems [T] and [2] follow from classical arguments in convex optimization. 

Proof of TheoremUl Consider a feasible X. Because A*X + Q € — 9/(Xo), 

/(X) - /(X ) > -(A*X + Q, X - X ) = -<Q, X) > 0, 

where the last equality uses «4(X) = ^4(Xo) and Q _L Xo, and the last equality uses Q ^ and 
X y 0. □ 

Proof of Theorem^ To prove (a) =>■ (b), we observe that ([3]) implies G <9x£(Xo, A, Q). Hence, 
Xo minimizes £(X, A, Q) over X. Hence g(A,Q) = £(Xo,A, Q). By slackness and feasibility of 
Xq, £(Xq, A, Q) = /(Xq) = p*. Hence (A, Q) is dual optimal and strong duality holds. 
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To prove (b) =^ (a), we observe that strong duality and dual optimality of (A, Q) imply 

/(X ) = inf /(X) + (A, A(X) -b) + (Q,X) (10) 

.X. 

In particular, /(Xo) < /(Xo) + (A, ^4(Xo) — b) + (Q,Xo), which implies (Q,Xo) > by feasibility 
of Xo. By dual feasibility, Q ■< and hence (Q,Xo) < 0. We thus have Q _L Xo- The inflmum in 
([10]) is achieved by X . Hence, <E <9x£(X , A, Q), and we conclude A*\ + Q G -<9/(X ). □ 

3 Proofs of Main Results 

In this section, we present the proofs of the main theorems and Corollary [TJ 
3.1 Proof of Theorems [3] and [4] and Corollary CO 

Under the assumptions of Theorem [3] the set S trivially satisfies the completeness condition ([8]) . 
The theorem is thus a special case of Theorem U] and we will prove them together. 

The strategy of proof involves rewriting ([!} in an unconstrained form. Existence of a dual 
certificate is guaranteed when the subdifferentials of the sum of two indicator functions is the sum 
of their respective subdifferentials. In Lemma[T] we use a separating hyperplane argument to prove 
additivity of these subdifferentials under the condition ([8]). 

Proof of Theorems & and [7] We first rewrite the problem ([1]) without constraints. Xo minimizes 
([T]) if and only if Xo minimizes the problem 

min /(X) + I X hCM(X)=b(X), (11) 

which, by convexity, happens if and only if 

0G5(/ + / x ^(x)=b)(Xo). (12) 

By assumption, / is continuous everywhere. Hence, the Moreau-Rockefellar Theorem [15] guaran- 
tees that 

d(f + /xMM(x)=b)(Xo) = 3/(X ) + <9/ X MM(x)=b(Xo). (13) 
By Lemma [H the completeness condition ([8]) is equivalent to 

9/xbo^(X)=b(X ) = 9/x b o(X ) + 0/A(x)=b(Xo). (14) 

We note that 

<9/ Xh o(X ) = {Q | Q r< 0,Q _L X }, (15) 

dI A[X)=h (X ) = {A*X}, (16) 

9/ Xb o(X ) + a/A(x)=b(Xo) = + Q | Q d 0, Q JL X } = S. (17) 

We conclude there exists a dual certificate (A, Q) by combining ([T2]) . ([13]) . ([H]). and (fT7]) . □ 

The corollary follows from Theorem [3] because the independence assumption implies that there 
are no nontrivial linear combinations of measurement matrices that are positive and orthogonal to 
Xo- 

Proof of Corollary Ul Consider A >z 0, A = ^AjAj, A _L x Xq. By ([0]), Ax = 0. Hence, 
J2i AjAjXo = 0. By independence of {AjXo}, Aj = for all i. Hence the conditions of Theorem [3] 
are met and there exists a dual certificate at Xo. 

□ 
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3.2 Proof of Theorem 02 



Theorem [5] provides a weak form of necessity for the completeness condition ([S]). If — <9/(Xo) only 
contains matrices that are not of form 5, there will be no dual certificate. When S does not satisfy 
([8]), there is a matrix orthogonal to all feasible points, and we choose / to have a gradient in the 
opposite direction. This argument also plays an important role in the primary technical lemma 
establishing equivalence between the completeness condition and additivity of subgradients. 

Proof of Theorem O If S does not satisfy the completeness condition ([8]) , then there is a q _L xo 
and a y such that qq* € S and y®q^5. Consider the problem 

min(-y <g> q, X) subject to X y 0, A(X) = A(X ). (18) 

Because qq* € S, qq* = .4* A + Q for some Q ^ 0, Q _L Xo. Hence, 

(X, qq*) = (X - X , qq*) = (X - X , „4*A + Q) = (X - X , Q) = (X, Q) < 0, 

where the third equality uses A(X. — Xo) = 0. Because feasibility of X implies (X, qq*) > 0, we 
conclude all feasible X _L qq*. Hence, (— y (g>q, X) = for any feasible X. Hence Xo is a minimizer. 
There is no dual certificate because — 5/(Xo) contains the single element y fi) q ^ S. □ 

4 Technical Lemmas 

The main technical lemma establishes additivity of subgradients of a class of indicator functions. 
One direction follows from the same argument as the proof of Theorem The other direction 
follows by showing Y ^ S =^ Y ^ <9-Tx>-o,.4(x)=b(Xo)- To show Y is not such a subgradient, we 
explicitly build a feasible X using a separating hyperplane argument. That argument requires S 
be closed, as proven in Lemma [2j It also hinges on Lemma [J] which classifies when perturbations 
from Xo remain positive semidefinite. 

Lemma 1. Let X = x Xq and A(X. ) = b. S = {A*\ + Q | Q H 0, Q _L X } satisfies the 
completeness condition ([8]) if and only if 

9/ Xho ^(x)=b(Xo) = dlxbo(Xo) + d/4 (X )=b(Xo). (19) 

Proof of LemmaUl First, we show — ■ dSJ) =4> -i (fT9|) . There exists q _L x such that qq* £ 5 but 
y <8> q ^ S for some y. Following the calculation in the proof of Theorem [51 all feasible X are 
orthogonal to y®q. Hence, y<g>q G 9/ X bO,^(X)=b(Xo), but y<g)q g S = dI^ h0 (X )+dI A ^ =h (X ). 
Next, we show (JSJ) (|19|) . One inclusion in (|19|) is automatic: 

<9/ X MM(x)=b(X ) = <9(/ Xb o + ^(X)=b)(X ) D 5/ Xb o(X ) + ^ (x)=b (X ). (20) 

To prove the other inclusion, we let Y ^ S = <9-Tx>o(Xo) + <9/4(x)=b(Xo)- We will show that 
Y ^ cM"x^o,.4(x)=b(Xo) by exhibiting a feasible X such that (Y,X — Xo) > 0. 

As we will prove in Lemma [21 © implies that S is closed. By the separating hyperplane 
theorem, for any Z ^ S, there exists a Az such that 

A(A Z ) = 0, (21) 

(A z , Q) < for all Q ^ 0, Q _L X , (22) 

(A Z ,M) = if ±MeS, (23) 

(A z ,Z)>0. (24) 
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We observe that (J22J) implies V X ±A Z h 0. 

Let B = {qq* | qq* _L Xo,qq* ^ S}. We will build a A satisfying the following properties: 

A(A) = 0, (25) 

(A, Q) < for all Q ^ 0, Q ± X , (26) 

(A,M) = if ±M e 5, (27) 

(A, qq*) > for all qq* € B. (28) 

We build A through the following process. Choose a qiq* € i? and find a corresponding A qiq *. 
Restrict B to a set -B containing only the elements that are orthogonal to A qiq *. All elements 
in B \ B have a positive inner product with A qiq *. Choose q2q<2 € B and find A q2q *. Further 
restrict B to only the elements that are orthogonal to A q2q *. Now, all elements in B \ B have a 
positive inner product with A qiq * or A q2q * . Repeat this process until B is empty. The process will 
complete after a finite number of repetitions because the set B is restricted to a space of strictly 
decreasing dimension at each step. Let A = J2i A qiq *. We observe ([2"5" ]) -([2"7 |l hold due to (f2T]) - (f2"3|) . 
Every element of B has a positive inner product with A q . q * for some i. Hence, we have (|28|) . 

Let A = AY + eA, where e is small enough that (A, Y) > 0. By Lemma HI if (a) V X ±A y 
and (b) A _L qq* and qq* _L Xo => A _L xo <8> q, then there exists 5 > such that Xo + <5A >; 0. 
By {22} and J2B]), (a) holds. To show (b) holds, we consider a qq* _L A, qq* _L X . By ([28]) and 
the definition of A, qq* must be in S. By (jSJ), ±x ® q € S. Hence, by ([23]) and ([2"T|). Alx ®q, 
and (b) holds. 

As given by Lemma [H let X = Xo + 5A. Because X y and .4(A) = 0, X is feasible. 
Additionally, (Y,X — Xo) > because (A, Y) > 0. Hence, Y ^ 97x^o,^(x)=b(^o)- 

□ 

The hyperplane separation argument above requires that S be closed. The following lemma 
reduces the closedness of S C Kgy^ toann — lxn — 1 case without the orthogonality constraint, 
which is proved in Lemma [3] 

Lemma 2. If S = A^Aj + Q j Q < 0, Q _L Xo} satisfies the completeness condition (|5]) then 
S is closed. 

Proof of Lemma\^ Without loss of generality let Xq = e\e\. This can be seen by letting V be an 
orthogonal matrix with xo/||xo|| in the first column, and by considering the set V*SV. If necessary, 
linearly recombine the A, such that the first columns of Ai, . . . , Ai are independent and the first 
columns of the remaining A^+i, . . . , A m are zero. 

Consider a Cauchy sequence A^ + Q( fc ) — > X, where A^ = Y^=i ^f^Ai- We will establish 
that X G S. Because Q( fc ) ^ and Q( fc ) _L eie*, it is zero in the first row and column. Hence the 
first column of Yli=i ^i-^i converges to the first column of X. By independence, we obtain that 
\\ k ^ converges to some X^ 00 ^ for each 1 < i < £. As a result, 

m 

Af } A, + Q( fc ) -> X, 

i=£+l 

where X = X — Y2i=i A^ Aj, and X is zero in the first row and column. 

The problem has now been reduced to one of size n — 1 x n — 1 without an orthogonality 
constraint, and Lemma [3] completes the proof. Let Aj be the lower-right n-lxn-1 sub matrix 
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of A,. Let S = {£Zt+i + Q I Q ^ 0} G Ksym Xn_1 - If qq* e S then Q Q G 5. By ©, 



0\ _ /0\ _ mft _! 



^ G 5 Vy G M n . By independence of the first columns of Ai, . . . , Ae, y <8> q G S. The 

conditions of Lemma El are met. Hence, X = YT=£+i >^ M + Q (oo) wi t h Q (oo) ^ 0, QH _L e x e\. 
We conclude X G S and S 1 is closed. 

□ 

The closedness of S above relies on the closedness of a lower dimensional S without the orthog- 
onality constraint. For any Cauchy sequence in S, we show the limit belongs to S. If there is no 
intersection between the range of A* and the space of negative semidefinite matrices, then these 
sets have a fixed angle between them. Hence, dual variables corresponding to a Cauchy sequence 
within S can not diverge separately yet converge when combined. If there is an intersection between 
the range of A* and the space of negative semidefinite matrices, the variant of the completeness 
condition allows us to project away from those directions. 

Lemma 3. The set S = {£V AjAj + Q | Q H 0} C R?™ is closed if 

qq* G S => y <8> q G S Vy. (29) 

Proof of Lemma\M Consider a Cauchy sequence A^ + — > X, where A^ = A^Aj. Let 
V = span{q | qq* G S}. For each q G V, (|29|) gives that y <g> q G S Vy. Because Vyx is the 
projection of matrices onto matrices with row and column spaces living in V , 

±(X - V V ±X) G S for any X. (30) 

The Cauchy sequence satisfies 

V V ±A^ + 7VQ (fc) Pi/iX. (31) 

Ttll „ .ft-),, ^ ||7VxAW|| F 1 , / V V± A^ V v± Q {k > \ , , 

If ||7VxA( fc )|| F -> oo, then ^q(^ -> 1 and ^ ' PVxQ w IIf / "1 as /c ^ oc. 

The sets {A G Pyxspan Aj} n {||A||p = 1} and {Q _< 0} n {||Q|[f = 1} are compact. Hence 
(A, Q) achieves its minimum. That minimum value must be —1, which implies that there exists a 
nonzero, positive semidefinite matrix — Q G "P^xspan Aj. This is is impossible by the construction 
of V. Suppose V V ±A*\ >z 0. By (f30j) . we see V V ±A*\ G S. Hence every rank-1 component qq* 
of Vy±A*X y belongs to S. We reach a contradiction because q would belong to V and can not 
be in the column space of V V ±A*\. 

Hence, V V ±A^ has a bounded subsequence. Thus, there is a further subsequence that con- 
verges and V v xX is of the form ?V(£i A^ Aj+Q(°°)). By ([30]), we conclude X = YT=i A-° c) A i + 
Q(°°) with Q(°°) < 0. 

□ 

The following lemma establishes a necessary and sufficient condition for when a symmetric 
perturbation from a positive rank 1 matrix remains positive. 

Lemma 4. Let Xo = xoXq G R nxn . Xo + SA y for some 5 > if and only if (a) V X ±A y and 
(b) A _L qq* and q _L xq => A _L xq <g> q. 
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Proof. Without loss of generality, assume Xo = e±e*. In this case V x ± is the restriction to the 
lower-right n-lxn-1 block. Let A x ± G M™^^™" 1 be that lower-right block of A. Write the 
block form 

'An P* 

p K 



A 



First we prove Xo + 5 A y => (a) and (b). We immediately have (a) because Xo is zero on 
the lower-right subblock. Using a Schur complement, 

if 1 + 6 A u > 0, then X + 5A y <^> A, — — pp* y 0. (32) 

o 1 + oAii 

If necessary, 5 can be reduced to enforce 1 + 6 An > 0. If (b) does not hold, then there is £ G M n_1 
such that A x x _L and p JL £. By testing against £, we see A x x — pp* ^ 

Second, we prove (a) and (b) => Xo + <5A for some <5 > 0. Assume (a) and (b) hold. Using the 
property ([32]) about Schur complements, it suffices to show 

A ^ - irkr""" - a (33) 

Let V = span {q | A x ± _L qq*} C M™ yn J x ™ _1 . There is some e such that A x ± y el v ±. If not, there 

would be a sequence of x^-* G such that ||x^|| = 1 and < x^ £ -'A x ±x( £ - ) * < e. Such would 

have a convergent subsequence to some x^ ) G V 1 - such that x(°)A x xx(°)* = 0, which is impossible 
by choice of V. 

We note that for any qGV, (b) guarantees p _L q. Hence p G and there is a sufficiently 
small 5 such that 1+ / A pp* _^ elyx. We conclude that (|33l) holds, and hence 35 > such that 
X + 5 A y 0. □ 
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